Augmented-reality-assisted intraoral scanning: A proof-of-concept study.

PURPOSE: The aims of the present study were (a) to compare the scanning time and image count to complete optical scans of a typodont between augmented-reality-assisted intraoral scanning (ARIOS) and intraoral scanning (IOS); (b) to compare the accuracy of the digital casts derived from ARIOS and IOS; (c) to compare participant-related outcomes between ARIOS and IOS. MATERIALS AND METHODS: A multi-session within-subject experiment was conducted to compare ARIOS and IOS. Thirty-one dental students participated in the study. Following a trial session, each participant obtained optical scans under ARIOS and IOS conditions. The time required to complete the scan, and the number of images taken were recorded. Participant feedback was collected using entry, exit, and NASA-Task Load Index (TLX) surveys. The accuracy of the digital casts derived from the optical scans was measured in root mean square error (RMSE). RESULTS: The present study found a 6.8% increase in preference for ARIOS from entry to exit survey. Slightly more participants favored the ARIOS setup compared to IOS; 54.8% of participants favored ARIOS, 9.7% were indifferent, and 35.5% favored IOS. NASA-TLX subscale ratings were higher for IOS in general apart from mental demand. The accuracy of the digital casts between ARIOS and IOS was comparable in RMSE. CONCLUSION: ARIOS was advantageous compared to IOS in ergonomics, improved scanner tracking, and ease of scanner orientation. However additional trials, increased field of view, and better superimposition of scanning status to the target site were improvements desired by the study participants.

Visuo-Haptic VR and AR Guidance for Dental Nerve Block Education.

The inferior alveolar nerve block (IANB) is a dental anesthetic injection that is critical to the performance of many dental procedures. Dental students typically learn to administer an IANB through videos and practice on silicone molds and, in many dental schools, on other students. This causes significant stress for both the students and their early patients. To reduce discomfort and improve clinical outcomes, we created an anatomically informed virtual reality headset-based educational system for the IANB. It combines a layered 3D anatomical model, dynamic visual guidance for syringe position and orientation, and active force feedback to emulate syringe interaction with tissue. A companion mobile augmented reality application allows students to step through a visualization of the procedure on a phone or tablet. We conducted a user study to determine the advantages of preclinical training with our IANB simulator. We found that in comparison to dental students who were exposed only to traditional supplementary study materials, dental students who used our IANB simulator were more confident administering their first clinical injections, had less need for syringe readjustments, and had greater success in numbing patients.

Multi-Level Precues for Guiding Tasks Within and Between Workspaces in Spatial Augmented Reality.

We explore Spatial Augmented Reality (SAR) precues (predictive cues) for procedural tasks within and between workspaces and for visualizing multiple upcoming steps in advance. We designed precues based on several factors: cue type, color transparency, and multi-level (number of precues). Precues were evaluated in a procedural task requiring the user to press buttons in three surrounding workspaces. Participants performed fastest in conditions where tasks were linked with line cues with different levels of color transparency. Precue performance was also affected by whether the next task was in the same workspace or a different one.

Mitigation of VR Sickness During Locomotion With a Motion-Based Dynamic Vision Modulator.

In virtual reality, VR sickness resulting from continuous locomotion via controllers or joysticks is still a significant problem. In this article, we present a set of algorithms to mitigate VR sickness that dynamically modulate the user's field of view by modifying the contrast of the periphery based on movement, color, and depth. In contrast with previous work, this vision modulator is a shader that is triggered by specific motions known to cause VR sickness, such as acceleration, strafing, and linear velocity. Moreover, the algorithm is governed by delta velocity, delta angle, and average color of the view. We ran two experiments with different washout periods to investigate the effectiveness of dynamic modulation on the symptoms of VR sickness, in which we compared this approach against a baseline and pitch-black field-of-view restrictors. Our first experiment made use of a just-noticeable-sickness design, which can be useful for building experiments with a short washout period.

Precueing Object Placement and Orientation for Manual Tasks in Augmented Reality.

When a user is performing a manual task, AR or VR can provide information about the current subtask (cueing) and upcoming subtasks (precueing) that makes them easier and faster to complete. Previous research on cueing and precueing in AR and VR has focused on path-following tasks requiring simple actions at each of a series of locations, such as pushing a button or just visiting. We consider a more complex task, whose subtasks involve moving to and picking up an item, moving that item to a designated place while rotating it to a specific angle, and depositing it. We conducted two user studies to examine how people accomplish this task while wearing an AR headset, guided by different visualizations that cue and precue movement and rotation. Participants performed best when given movement information for two successive subtasks and rotation information for a single subtask. In addition, participants performed best when the rotation visualization was split across the manipulated object and its destination.

Using Multi-Level Precueing to Improve Performance in Path-Following Tasks in Virtual Reality.

Work on VR and AR task interaction and visualization paradigms has typically focused on providing information about the current step (a cue) immediately before or during its performance. Some research has also shown benefits to simultaneously providing information about the next step (a precue). We explore whether it would be possible to improve efficiency by precueing information about multiple upcoming steps before completing the current step. To accomplish this, we developed a remote VR user study comparing task completion time and subjective metrics for different levels and styles of precueing in a path-following task. Our visualizations vary the precueing level (number of steps precued in advance) and style (whether the path to a target is communicated through a line to the target, and whether the place of a target is communicated through graphics at the target). Participants in our study performed best when given two to three precues for visualizations using lines to show the path to targets. However, performance degraded when four precues were used. On the other hand, participants performed best with only one precue for visualizations without lines, showing only the places of targets, and performance degraded when a second precue was given. In addition, participants performed better using visualizations with lines than ones without lines.

Leveraging Patient-Reported Outcomes Using Data Visualization.

BACKGROUND: Health care organizations increasingly use patient-reported outcomes (PROs) to capture patients' health status. Although federal policy mandates PRO collection, the challenge remains to better engage patients in PRO surveys, and ensure patients comprehend the surveys and their results. OBJECTIVE: This article identifies the design requirements for an interface that assists patients with PRO survey completion and interpretation, and then builds and evaluates the interface. METHODS: We employed a user-centered design process that consisted of three stages. First, we conducted qualitative interviews and surveys with 13 patients and 11 health care providers to understand their perceptions of the value and challenges associated with the use of PRO measures. Second, we used the results to identify design requirements for an interface that collects PROs, and designed the interface. Third, we conducted usability testing with 12 additional patients in a hospital setting. RESULTS: In interviews, patients and providers reported that PRO surveys help patients to reflect on their symptoms, potentially identifying new opportunities for improved care. However, 6 out of 13 patients reported significant difficultly in understanding PRO survey questions, answer choices and results. Therefore, we identified aiding comprehension as a key design requirement, and incorporated visualizations into our interface design to aid comprehension. In usability testing, patients found the interface highly usable. CONCLUSION: Future interfaces designed to collect PROs may benefit from employing strategies such as visualization to aid comprehension and engage patients with surveys.

Patient Experiences Using an Inpatient Personal Health Record.

OBJECTIVE: To investigate patients' experience using an inpatient personal health record (PHR) on a tablet computer to increase engagement in their hospital care. METHODS: We performed observations and conducted semi-structured interviews with 14 post-operative cardiac surgical patients and their family members who received an inpatient PHR. Themes were identified using an inductive coding scheme. RESULTS: All participants responded favorably to having access to view their clinical information. A majority (85.7%) of participants used the application following an initial training session. Patients reported high satisfaction with being able to view their hospital medications and access educational materials related to their medical conditions. Patients reported a desire to view daily progress reports about their hospital stay and have access to educational information about their post-acute recovery. In addition, patients expressed a common desire to view their diagnoses, laboratory test results, radiology reports, and procedure notes in language that is patient-friendly. CONCLUSION: Patients have unmet information needs in the hospital setting. Our findings suggest that for some inpatients and their family members, providing personalized health information through a tablet computer may improve satisfaction, decrease anxiety, increase understanding of their health conditions, and improve safety and quality of care.

Interactive tools for inpatient medication tracking: a multi-phase study with cardiothoracic surgery patients.

OBJECTIVE: Prior studies of computing applications that support patients' medication knowledge and self-management offer valuable insights into effective application design, but do not address inpatient settings. This study is the first to explore the design and usefulness of patient-facing tools supporting inpatient medication management and tracking. MATERIALS AND METHODS: We designed myNYP Inpatient, a custom personal health record application, through an iterative, user-centered approach. Medication-tracking tools in myNYP Inpatient include interactive views of home and hospital medication data and features for commenting on these data. In a two-phase pilot study, patients used the tools during cardiothoracic postoperative care at Columbia University Medical Center. In Phase One, we provided 20 patients with the application for 24-48 h and conducted a closing interview after this period. In Phase Two, we conducted semi-structured interviews with 12 patients and 5 clinical pharmacists who evaluated refinements to the tools based on the feedback received during Phase One. RESULTS: Patients reported that the medication-tracking tools were useful. During Phase One, 14 of the 20 participants used the tools actively, to review medication lists and log comments and questions about their medications. Patients' interview responses and audit logs revealed that they made frequent use of the hospital medications feature and found electronic reporting of questions and comments useful. We also uncovered important considerations for subsequent design of such tools. In Phase Two, the patients and pharmacists participating in the study confirmed the usability and usefulness of the refined tools. CONCLUSIONS: Inpatient medication-tracking tools, when designed to meet patients' needs, can play an important role in fostering patient participation in their own care and patient-provider communication during a hospital stay.

Engaging hospitalized patients in clinical care: Study protocol for a pragmatic randomized controlled trial.

BACKGROUND: Patients who are better informed and more engaged in their health care have higher satisfaction with health care and better health outcomes. While patient engagement has been a focus in the outpatient setting, strategies to engage inpatients in their care have not been well studied. We are undertaking a study to assess how patients' information needs during hospitalization can be addressed with health information technologies. To achieve this aim, we developed a personalized inpatient portal that allows patients to see who is on their care team, monitor their vital signs, review medications being administered, review current and historical lab and test results, confirm allergies, document pain scores and send questions and comments to inpatient care providers. The purpose of this paper is to describe the protocol for the study. METHODS/DESIGN: This pragmatic randomized controlled trial will enroll 426 inpatient cardiology patients at an urban academic medical center into one of three arms receiving: 1) usual care, 2) iPad with general internet access, or 3) iPad with access to the personalized inpatient portal. The primary outcome of this trial is patient engagement, which is measured through the Patient Activation Measure. To assess scalability and potential reach of the intervention, we are partnering with a West Coast community hospital to deploy the patient engagement technology in their environment with an additional 160 participants. CONCLUSION: This study employs a pragmatic randomized control trial design to test whether a personalized inpatient portal will improve patient engagement. If the study is successful, continuing advances in mobile computing technology should make these types of interventions available in a variety of clinical care delivery settings.

Attributes of subtle cues for facilitating visual search in augmented reality.

Goal-oriented visual search is performed when a person intentionally seeks a target in the visual environment. In augmented reality (AR) environments, visual search can be facilitated by augmenting virtual cues in the person's field of view. Traditional use of explicit AR cues can potentially degrade visual search performance due to the creation of distortions in the scene. An alternative to explicit cueing, known as subtle cueing, has been proposed as a clutter-neutral method to enhance visual search in video-see-through AR. However, the effects of subtle cueing are still not well understood, and more research is required to determine the optimal methods of applying subtle cueing in AR. We performed two experiments to investigate the variables of scene clutter, subtle cue opacity, size, and shape on visual search performance. We introduce a novel method of experimentally manipulating the scene clutter variable in a natural scene while controlling for other variables. The findings provide supporting evidence for the subtlety of the cue, and show that the clutter conditions of the scene can be used both as a global classifier, as well as a local performance measure.

Patient engagement in the inpatient setting: a systematic review.

OBJECTIVE: To systematically review existing literature regarding patient engagement technologies used in the inpatient setting. METHODS: PubMed, Association for Computing Machinery (ACM) Digital Library, Institute of Electrical and Electronics Engineers (IEEE) Xplore, and Cochrane databases were searched for studies that discussed patient engagement ('self-efficacy', 'patient empowerment', 'patient activation', or 'patient engagement'), (2) involved health information technology ('technology', 'games', 'electronic health record', 'electronic medical record', or 'personal health record'), and (3) took place in the inpatient setting ('inpatient' or 'hospital'). Only English language studies were reviewed. RESULTS: 17 articles were identified describing the topic of inpatient patient engagement. A few articles identified design requirements for inpatient engagement technology. The remainder described interventions, which we grouped into five categories: entertainment, generic health information delivery, patient-specific information delivery, advanced communication tools, and personalized decision support. CONCLUSIONS: Examination of the current literature shows there are considerable gaps in knowledge regarding patient engagement in the hospital setting and inconsistent use of terminology regarding patient engagement overall. Research on inpatient engagement technologies has been limited, especially concerning the impact on health outcomes and cost-effectiveness.

Patient-Centered Tools for Medication Information Search.

Recent research focused on online health information seeking highlights a heavy reliance on general-purpose search engines. However, current general-purpose search interfaces do not necessarily provide adequate support for non-experts in identifying suitable sources of health information. Popular search engines have recently introduced search tools in their user interfaces for a range of topics. In this work, we explore how such tools can support non-expert, patient-centered health information search. Scoping the current work to medication-related search, we report on findings from a formative study focused on the design of patient-centered, medication-information search tools. Our study included qualitative interviews with patients, family members, and domain experts, as well as observations of their use of Remedy, a technology probe embodying a set of search tools. Post-operative cardiothoracic surgery patients and their visiting family members used the tools to find information about their hospital medications and were interviewed before and after their use. Domain experts conducted similar search tasks and provided qualitative feedback on their preferences and recommendations for designing these tools. Findings from our study suggest the importance of four valuation principles underlying our tools: credibility, readability, consumer perspective, and topical relevance.

Designing Inpatient Technology to Meet the Medication Information Needs of Cardiology Patients.

As patients are encouraged to become active participants in their own care, recent research has begun to explore the direct sharing of electronic health information with patients during hospital visits. The design of patient-facing views of clinical information is, however, a relatively recent line of inquiry. Research is needed to further understand guidelines for communicating specific types of information to hospital patients. In this work, we focus on cardiology patients' information needs related to their hospital medications. We assessed these needs to inform the design of interactive, electronic views of medication information for cardiology inpatients. We present results of in-situ interviews with 11 inpatients and 6 nurses in a cardiology step-down unit. Our findings suggest that cohesive trends in medication information needs exist across cardiology inpatients. We discuss interview results and their implications for the design of inpatient-facing information technology. We also discuss key ways in which electronic medication information, formatted for inpatient use, differs from that formatted for outpatient or transitional medication-management use.

Exploring the benefits of augmented reality documentation for maintenance and repair.

We explore the development of an experimental augmented reality application that provides benefits to professional mechanics performing maintenance and repair tasks in a field setting. We developed a prototype that supports military mechanics conducting routine maintenance tasks inside an armored vehicle turret, and evaluated it with a user study. Our prototype uses a tracked headworn display to augment a mechanic's natural view with text, labels, arrows, and animated sequences designed to facilitate task comprehension, localization, and execution. A within-subject controlled user study examined professional military mechanics using our system to complete 18 common tasks under field conditions. These tasks included installing and removing fasteners and indicator lights, and connecting cables, all within the cramped interior of an armored personnel carrier turret. An augmented reality condition was tested against two baseline conditions: the same headworn display providing untracked text and graphics and a fixed flat panel display representing an improved version of the laptop-based documentation currently employed in practice. The augmented reality condition allowed mechanics to locate tasks more quickly than when using either baseline, and in some instances, resulted in less overall head movement. A qualitative survey showed that mechanics found the augmented reality condition intuitive and satisfying for the tested sequence of tasks.

Awareness of the Care Team in Electronic Health Records.

OBJECTIVE: To support collaboration and clinician-targeted decision support, electronic health records (EHRs) must contain accurate information about patients' care providers. The objective of this study was to evaluate two approaches for care provider identification employed within a commercial EHR at a large academic medical center. METHODS: We performed a retrospective review of EHR data for 121 patients in two cardiology wards during a four-week period. System audit logs of chart accesses were analyzed to identify the clinicians who were likely participating in the patients' hospital care. The audit log data were compared with two functions in the EHR for documenting care team membership: 1) a vendor-supplied module called "Care Providers", and 2) a custom "Designate Provider" order that was created primarily to improve accuracy of the attending physician of record documentation. RESULTS: For patients with a 3-5 day hospital stay, an average of 30.8 clinicians accessed the electronic chart, including 10.2 nurses, 1.4 attending physicians, 2.3 residents, and 5.4 physician assistants. The Care Providers module identified 2.7 clinicians/patient (1.8 attending physicians and 0.9 nurses). The Designate Provider order identified 2.1 clinicians/patient (1.1 attending physicians, 0.2 resident physicians, and 0.8 physician assistants). Information about other members of patients' care teams (social workers, dietitians, pharmacists, etc.) was absent. CONCLUSIONS: The two methods for specifying care team information failed to identify numerous individuals involved in patients' care, suggesting that commercial EHRs may not provide adequate tools for care team designation. Improvements to EHR tools could foster greater collaboration among care teams and reduce communication-related risks to patient safety.

Opportunistic tangible user interfaces for augmented reality.

Opportunistic Controls are a class of user interaction techniques that we have developed for augmented reality (AR) applications to support gesturing on, and receiving feedback from, otherwise unused affordances already present in the domain environment. By leveraging characteristics of these affordances to provide passive haptics that ease gesture input, Opportunistic Controls simplify gesture recognition, and provide tangible feedback to the user. In this approach, 3D widgets are tightly coupled with affordances to provide visual feedback and hints about the functionality of the control. For example, a set of buttons can be mapped to existing tactile features on domain objects. We describe examples of Opportunistic Controls that we have designed and implemented using optical marker tracking, combined with appearance-based gesture recognition. We present the results of two user studies. In the first, participants performed a simulated maintenance inspection of an aircraft engine using a set of virtual buttons implemented both as Opportunistic Controls and using simpler passive haptics. Opportunistic Controls allowed participants to complete their tasks significantly faster and were preferred over the baseline technique. In the second, participants proposed and demonstrated user interfaces incorporating Opportunistic Controls for two domains, allowing us to gain additional insights into how user interfaces featuring Opportunistic Controls might be designed.

Physician-Driven Management of Patient Progress Notes in an Intensive Care Unit.

We describe fieldwork in which we studied hospital ICU physicians and their strategies and documentation aids for composing patient progress notes. We then present a clinical documentation prototype, activeNotes, that supports the creation of these notes, using techniques designed based on our fieldwork. ActiveNotes integrates automated, context-sensitive patient data retrieval, and user control of automated data updates and alerts via tagging, into the documentation process. We performed a qualitative study of activeNotes with 15 physicians at the hospital to explore the utility of our information retrieval and tagging techniques. The physicians indicated their desire to use tags for a number of purposes, some of them extensions to what we intended, and others new to us and unexplored in other systems of which we are aware. We discuss the physicians' responses to our prototype and distill several of their proposed uses of tags: to assist in note content management, communication with other clinicians, and care delivery.

Physician Attitudes about Patient-Facing Information Displays at an Urban Emergency Department.

Hospital information systems have primarily been designed to support physicians and administrators, though recent research has explored the value of patient-facing information displays. Electronic systems can be designed to provide tailored information to patients on their health, their care teams, the status of their hospital stays, and their expected care plans. However, this direct delivery of information from database to patient represents a fundamental change to the traditional flow of clinical information. We therefore explore physician attitudes toward a proposed patient-facing display of information abstracted from a hospital EHR, in the context of an urban emergency department. We find that physicians generally support direct delivery of electronic information to patients, and uncover important concerns to consider in the design of patient-facing information systems.

activeNotes: Computer-Assisted Creation of Patient Progress Notes.

We present activeNotes, a prototype application that supports the creation of Critical Care Notes by physicians in a hospital intensive care unit. activeNotes integrates automated, context-sensitive patient data retrieval and user control of automated data updates and alerts into the note-creation process. In a user study at New York Presbyterian Hospital, we gathered qualitative feedback on the prototype from 15 physicians. The physicians found activeNotes to be valuable and said they would use it to create both formal notes for medical records and informal notes. One surprising finding is that while physicians have rejected template-based clinical documentation systems in the past, they expressed a desire to use activeNotes to create personalized, physician-specific note templates to be reused with a given patient, or for a given condition.